Summary

Fixes the v0.2.4 stack-budget regression on UniFi 5.15 kernels by splitting the BPF datapath into two programs connected by bpf_tail_call. Each program gets its own 512-byte stack budget. Same architecture establishes the pattern for future fast-path-internal stages (additional packet transforms, more sophisticated FIB logic) without re-bisecting stack bytes every release.

The proximate failure (reported on edge1-mci1-net): UniFi's 5.15.72-ui-cn9670 aarch64 kernel rejected v0.2.4's fast_path BPF program at combined stack size of 3 calls is 544. Too large — same bytecode that loaded cleanly on CI's qemu 5.15 vanilla x86_64 (stack depth 0+360+0+0). UniFi's BPF patches plus the aarch64 JIT account stack ~120 bytes higher than vanilla.

Architecture

                       packet ingress
                              │
                              ▼
   ┌──────────────────────────────────────────────────┐
   │ fast_path  (XDP, attached to eth0..ethN)         │  Frame A
   │   classification (allow-prefix, block-prefix)     │  fits 512B
   │   FIB lookup  (kernel-fib | custom-fib | compare)│
   │   devmap pre-check                                │
   │   TTL decrement / L2 rewrite (in-place)           │
   │   write per-CPU MUTATION_CTX                      │
   │   bpf_tail_call(MUTATION_PROGS, 0)  ──────────┐  │
   └────────────────────────────────────────────────│──┘
                                                    │
                                                    ▼
   ┌──────────────────────────────────────────────────┐
   │ finalize  (XDP, tail-called by fast_path)        │  Frame B
   │   read MUTATION_CTX                               │  fresh 512B
   │   mss-clamp lookup + (optional) MSS rewrite       │
   │   VLAN choreography (push / pop / rewrite)        │
   │   bpf_redirect_map(egress_ifindex)                │
   └──────────────────────────────────────────────────┘

mss-clamp + VLAN + redirect move from forward_success into finalize. Per-prefix LPM keys + TCP-options walk live in finalize's fresh stack budget. fast_path's responsibilities shrink to classification + L2/TTL, which fits comfortably under any kernel's accounting.

This is not the multi-module dispatcher (SPEC §3.4 / §5.0) — that's for chaining independent modules at the same hook (ddos in front of fast-path, sampler behind it). Tail-call is for splitting one logical pipeline. Both eventually exist; v0.2.5 ships only the former.

What's in the PR

What's deliberately NOT in this PR

• Netns end-to-end integration test (tests/tail_call.rs was in the plan). The kernel BPF_PROG_TEST_RUN harness already exercises the tail-call via the existing fixtures (now updated to populate MUTATION_PROGS[0]). A real-veth + AF_PACKET capture test is good additional coverage but ~150 LOC of test infra and was the right thing to defer to keep this PR focused on the architecture change. Will land as a follow-up.
• bpf_fib_lookup per-CPU map move (mentioned as "alternative B" in earlier discussion). Tail-call obviates it; we have plenty of stack headroom now.
• Multi-stage chain (slots 1-7 of MUTATION_PROGS). Reserved capacity is in place for future use; no actual stages today.
• The dispatcher (SPEC §5.0). Different problem entirely — for ddos / sampler / randomizer composition.

CI expectations

Existing CI matrix should pass:

• ✅ fmt + clippy + test (workspace lib tests pass on macOS dev: 94 + 40)
• ✅ Cross-build matrix (4 targets) — userspace-only, no BPF concern
• ✅ qemu-verifier 5.15 + 6.6 — should load both programs, attach succeeds, sudo-gated test fixtures pass through the tail-call chain

The qemu jobs are the meaningful test for "does the verifier accept this on real kernels." If they pass, vanilla 5.15 / 6.6 are fine. UniFi-style stricter accounting will be confirmed via post-merge deployment on the same router that hit the v0.2.4 regression.

Test plan

Pre-merge (CI):

• cargo fmt --all --check clean
• cargo clippy --workspace --all-targets --all-features -- -D warnings clean
• cargo test --workspace --lib 94 + 40 pass on macOS dev host
• CI fmt+clippy+test passes
• CI cross-build (4 targets) passes
• CI qemu-verifier 5.15 + 6.6 passes (with updated harness following the chain)

Post-merge on the deployed UniFi router:

• apt install ./packetframe_0.2.5_arm64.deb. sudo systemctl restart packetframe.
• sudo packetframe feasibility --config /etc/packetframe/packetframe.conf --human — all xdp.attach.ethN now PASS (no more 544/512 rejection).
• sudo packetframe status — the new "tail-call chain" section reports MUTATION_PROGS[0]: populated (finalize).
• Add a per-prefix mss-clamp directive: mss-clamp 23.191.200.0/24 1360. sudo packetframe reconfigure.
• tcpdump -i eth2 -n 'tcp[tcpflags] & tcp-syn != 0' -vv confirms wire MSS=1360 on outbound SYNs.
• sudo packetframe status | grep mss_clamp_applied shows the counter climbing.
• err_tail_call and err_mutation_ctx stay at 0.

Tag flow after merge

git checkout main && git pull
git tag -a v0.2.5 -m "v0.2.5"
git push origin v0.2.5

(The version bump is in this PR — same pattern as v0.2.4 — so just tag and push.)

🤖 Generated with Claude Code